Filtering and stochastic control: A historical perspective

I n this article we attempt to give a historical account of the main ideas leading to the development of non-linear filtering and stochastic control as we know it today. The article contains six sections. In the next section we present a development of linear filtering theory, beginning with Wiener-Kolmogoroff filtering and ending with Kalman filtering. The method of development is the innovations method as origi nally proposed hy Roue and Shannon and later presenled in ils modern form by Kailath. The third section is concerned with the Linear-Quadratic-Gaussian problem of stochastic control. Here we give a discussion of the separation theorem which states that for this problem the optimal stochastic control can be constructed by solving separately a state estimation problem and a determi nistic optimal control problem. Many of the ideas presented here generalize to the non-linear situation. The fourth section gives a reasonably detailed discussion of non-linear filtering, again from the innovations viewpoint. Finally, the fifth and sixth sections are concerned with optimal stochastic control. The general method of discussing these problems is Dynamic Programming. We have chosen to develop the subject in continuous time. In order to obtain correct results for nonlinear stochastic problems in continuous time it is essential that the modern language and theory of stochastic processes and stochastic differential equa tions be used. The book of Wong [5] is the preferred text. Some of this language is summarized in the third section. Wiener and Kalman Filtering In order to introduce the main ideas of non-linear filtering we first consider linear filtering theory. A rather comprehensive survey of linear filtering theory was undertaken by Kailath in [1] and therefore we shall only expose those ideas which generalize to the non-linear situation. Suppose we have a signal process (Zt) and an orthogonal increment process (w,), the noise process and we have the observation equation where 111 is the formal (distributional) derivative of Brownian motion and hence it is white noise. We make the following assumptions. (AI) (Wt) has stationary orthogonal increments (A2) (Zt) is a second-order q.m. continuous process (A3) For 'ds and t> s where H;"z is the Hilbert space spanned by (w�, z� I T:S: s). The last assumption is a causality requirement but includes situations where the signal Zs may be influenced by past obser vations as would typically arise in feedback control problems. A slightly stronger assumption is (A3)' H W ..1 H Z which states that the signal and noise are uncorrelated, a situation which often arises in communication problems. The situation which Wiener considered corresponds to (2), where he assumed that (Zt) is a stationary, second-order, q.m. continuous process. The filtering problem is to obtain the best linear estimate z/ of Zt based on the past observations (Ys Iss t). There are two other problems of interest, namely, prediction, when we are interested in the best linear estimate zr' r> t based on observations (ys I s s t) and smoothing, where we require obtaining the best linear estimate z r ' r < t based on observations (ys Iss t). Abstractly, the solution to the problem of filtering corresponds to explicitly computing (3) where p,Y is the projection operator onto the Hilbert space Hi. We proceed to outline the solution using a method originally proposed by Bode and Shannon l2 J and later presented in modern form by Kailath [3]. For a textbook account see Davis Of fice under grant number DAAL03-92-G-01J5 (through the Center The following facts about the innovations process can be for Intelligent Control Systems)

A note on stochastic dissipativeness

In this paper we present a stochastic version of Willems'ideas on Dissipativity and generalize the dissipation inequality to Markov Diffusion Processes. We show the relevance of these ideas by examining the problem of Ergodic Control of partially observed diffusions

Volatility of Power Grids under Real-Time Pricing

The paper proposes a framework for modeling and analysis of the dynamics of supply, demand, and clearing prices in power system with real-time retail pricing and information asymmetry. Real-time retail pricing is characterized by passing on the real-time wholesale electricity prices to the end consumers, and is shown to create a closed-loop feedback system between the physical layer and the market layer of the power system. In the absence of a carefully designed control law, such direct feedback between the two layers could increase volatility and lower the system's robustness to uncertainty in demand and generation. A new notion of generalized price-elasticity is introduced, and it is shown that price volatility can be characterized in terms of the system's maximal relative price elasticity, defined as the maximal ratio of the generalized price-elasticity of consumers to that of the producers. As this ratio increases, the system becomes more volatile, and eventually, unstable. As new demand response technologies and distributed storage increase the price-elasticity of demand, the architecture under examination is likely to lead to increased volatility and possibly instability. This highlights the need for assessing architecture systematically and in advance, in order to optimally strike the trade-offs between volatility, economic efficiency, and system reliability

Channels That Die

Given the possibility of communication systems failing catastrophically, we investigate limits to communicating over channels that fail at random times. These channels are finite-state semi-Markov channels. We show that communication with arbitrarily small probability of error is not possible. Making use of results in finite blocklength channel coding, we determine sequences of blocklengths that optimize transmission volume communicated at fixed maximum message error probabilities. We provide a partial ordering of communication channels. A dynamic programming formulation is used to show the structural result that channel state feedback does not improve performance

Maximum work extraction and implementation costs for nonequilibrium Maxwell's demons

We determine the maximum amount of work extractable in finite time by a demon performing continuous measurements on a quadratic Hamiltonian system subjected to thermal fluctuations, in terms of the information extracted from the system. The maximum work demon is found to apply a high-gain continuous feedback involving a Kalman-Bucy estimate of the system state and operates in nonequilibrium. A simple and concrete electrical implementation of the feedback protocol is proposed, which allows for analytic expressions of the flows of energy, entropy, and information inside the demon. This let us show that any implementation of the demon must necessarily include an external power source, which we prove both from classical thermodynamics arguments and from a version of Landauer's memory erasure argument extended to nonequilibrium linear systems

On an Information and Control Architecture for Future Electric Energy Systems

This paper presents considerations towards an information and control architecture for future electric energy systems driven by massive changes resulting from the societal goals of decarbonization and electrification. This paper describes the new requirements and challenges of an extended information and control architecture that need to be addressed for continued reliable delivery of electricity. It identifies several new actionable information and control loops, along with their spatial and temporal scales of operation, which can together meet the needs of future grids and enable deep decarbonization of the electricity sector. The present architecture of electric power grids designed in a different era is thereby extensible to allow the incorporation of increased renewables and other emerging electric loads.Comment: This paper is accepted, to appear in the Proceedings of the IEE